System and method for iris data acquisition for biometric identification

ABSTRACT

A system and related method for acquiring high quality images of the iris of an unconstrained subject comprising a camera; a controllable focusing component; a focus controller component that controls the lens to focus at successively different points within a focus range, such focus control performed without any input from measurement of whether the image is in focus or out of focus, be it based from measurements of the image or other distance metrics to the subject; and a sharpness detection component that rejects the most out-of-focus images based on measurement of focus on the image is disclosed.

RELATED APPLICATIONS

This application is a continuation of and claims priority from U.S.application Ser. No. 12/675,189, entitled “SYSTEM AND METHOD FOR IRISDATA ACQUISITION FOR BIOMETRIC IDENTIFICATION”, which is a nationalstage entry of International application PCT/US08/74737, filed Aug. 29,2008, entitled “SYSTEM AND METHOD FOR IRIS DATA ACQUISITION FORBIOMETRIC IDENTIFICATION”, which claims priority from U.S. provisionalapplication 60/969,607, filed Sep. 1, 2007, entitled “METHODOLOGY FORACQUIRING BIOMETRIC DATA LARGE VOLUMES”, all of which are herebyincorporated by reference for all purposes.

BACKGROUND

This disclosure relates to systems and methods for acquiring biometricand other imagery, biometric acquisition, identification, frauddetection, and security systems and methods, particularly biometricsystems and methods which employ iris recognition. More particularly thedisclosure relates to systems and methods for acquiring iris data foriris recognition.

Iris recognition systems have been in use for some time. The acquisitionof images suitable for iris recognition is inherently a challengingproblem. The performance of recognition algorithms depends on thequality, i.e., sharpness and contrast, of the image of the iris of thesubject who is to be identified. This is due to many reasons. As anexample, the iris itself is relatively small (approximately 1 cm indiameter) and it is often required to observe it from a great distancein order to avoid constraining the position of the subject or when thesubject is walking or riding. This results in a small field of view andalso a small depth of field. As a second example, it is generallydifficult for the adult or child subject to stay absolutely still. As athird example, the subject may blink involuntarily or drop or swiveltheir head momentarily to check on the whereabouts of luggage.

In biometric identification applications, due to unconstrained motion ofcooperative or non-compliant subject, it has been very difficult toacquire iris images with sufficient quality for recognition andidentification processing. For example, iris acquisition systemstypically check whether the quality of an acquired image exceeds athreshold. Many methods of assessing quality have been developed, suchas those based on a measurement of focus such as those disclosed in U.S.Pat. No. 6,753,919. The problem with this approach is that if theacquired image quality does not exceed the threshold, then the data isnot acquired, despite the fact that there may never be anotheropportunity to acquire data from that subject again. More specifically,in the case of unconstrained users or non-cooperative subjects, it maybe impossible to have the subject position themselves or wait until theacquired image data exceeds the quality threshold. For example, thesubject may be distracted with their head turning in various directions,or they may be in the process of performing another task, such asboarding a bus, so that the opportunity to acquire data from them hasalready come and gone. More specifically, prior iris data acquisitionsystems have typically been designed to explicitly avoid capturing lowerquality data with an emphasis on waiting or constraining the user suchthat only highest quality data is acquired. We have determined that evena lower quality iris image (blurred, for example) can still containsubstantial evidence for matching, albeit not with the precision of ahigh quality iris image. However, we still wish to acquire high qualitydata when it is possible to do so. In another example of prior systems,for example those disclosed in U.S. Pat. No. 5,151,583, autofocusroutines are used to attempt to obtain high quality iris images.However, autofocus routines cause lag times and inaccuracy, resulting inpoor quality or even non-existent imaging. Other systems, such as theones disclosed in U.S. Pat. No. 6,753,919 by Daugman, use sensors toassist a subject in aligning and focusing a handheld video camera.

Most if not all automatic focus systems work by acquiring an image ofthe scene, processing the image to recover a measure of focus, usingthat measure of focus to move a lens-focus actuator, and then repeatingthe steps of image acquisition, processing and actuation many timesuntil it is determined in the processing step that focus has beenreached. In most iris recognition systems autofocus never is able tocatch up with the actual position of the subject unless the subject isrelatively stationary, due to the unusually low depth of field in irisrecognition, as well as the requirement that the focus has to be on theiris (as opposed to the nose for example).

Because of the time delays involved in acquiring an image, processingthe image, and mechanical actuation, it is impossible for auto-focusalgorithms to respond instantaneously. Moreover, as the depth of fieldreduces, as is typically the case in iris recognition, where the objectis small and is typically observed at high magnification, it becomesmore difficult for auto-focus algorithms to be successful because anyerror in the auto-focus position is much more apparent in the imagerysince the depth of field is small.

It is much more difficult for auto-focus to acquire in-focus imagery ofa subject who is moving even slightly (fractions of an inch).

In the case of a person moving even slightly because there is a finitecontrol loop time for standard auto-focus to actuate, it can be shownthat if a component of the person's motion is high frequency and abovethe control loop response time, then the auto-focus will never be ableto converge and acquire an in-focus image of the person. The auto-focuswill be continually “hunting” for a focused image and will always lagthe motion of the subject. The result is that the subject has to be rocksolid and still when standard auto-focus is used, and this was the stateof the art in iris recognition before the present invention.

Prior attempts to solve these autofocus problems use the same closedloop approach but assume a subject is moving in a straight line and thenuse the image measurements to try and predict where the person will bein the next frame. This approach is not very robust and also fails forrandom movement that subjects often have. Other auto-focus systems usedifferent ways of computing focus measures in the scene in one or moreregions to compute the most accurate focus score. When a subject ismoving with frequencies that are beyond the control loop of anauto-focus algorithm auto-focus algorithms are unable to catch up to theperson's motion and acquire a good image of the person.

Martin, et al., US Pat. Pub. 2008/0075335, disclose a biometric imageselection method which reduces the rate of non-exploitable images whichare supplied to an analysis and identification processing module usingsharpness and contrast criteria. In some embodiments Martin et al.locate a pattern in each image of a sequence of images, estimate thespeed of displacement of the pattern between two successive images inthe sequence, and select images for which the estimated speed ofdisplacement of the pattern is lower than a speed threshold. Martin etal. disclosed embodiments wherein two selection modules are provided,the first being a quick selection module and the second being a pupiltracking module, rejecting an image if it is below a contrast orsharpness threshold. The selection module in some embodiments selectsimages having the highest sharpness and/or contrast out of the imagesstored. Martin et al do not disclose a system or method for acquiringthe series of images, nor do they disclose storing only images havinghigher quality than previously stored images and removing the lesserquality image from memory storage.

SUMMARY

The foregoing disadvantages and problems are overcome by the presentinvention which automatically acquires a series of images, analyzes theimages for quality, and stores only the best quality image, notnecessarily dependent on whether the quality exceeds a predeterminedthreshold, thereby saving memory and assuring that at least one image isstored, even if not having a quality exceeding a threshold. In a secondembodiment, the system which does not require an auto-focusing systembut rather automatically acquires a series of images at different focussettings regardless of the quality of images previously acquired,analyzes the images for quality, and stores only the best quality image,not necessarily dependent on whether the quality exceeds a predeterminedthreshold, thereby saving memory and assuring that at least one image isstored, even if not having a quality exceeding a threshold. Theinvention is an iris image acquisition system that, over the smallestpossible time period for a particular subject, stores successivelybetter quality images of the iris among the images acquired by theacquisition system to ensure that at least some biometric data of thesubject is acquired, while at the same time accounting for arbitrary andrapid subject motion, and voluntary or involuntary subject actions suchas, for example, eye blinks or head twists, all with a minimal memoryrequirement.

The invention is directed to acquiring iris images of optimum qualityfor further processing which comprises matching iris images of unknownsubjects to iris image templates of known subjects. In another aspectthe invention comprises a system and method of acquiring iris imageshaving the best focus without use of autofocus systems or methods. Inanother aspect the invention comprises a method of acquiring iris imagescomprising deploying a lens with a controllable adjustable focus; andadjusting focus without feedback from a focus measurement value. In someembodiments the lens is scanned over a range of focus values. The systemof the invention controls the lens to have an opportunistic capturewhich scans through different slices of depth volume, acquiring data.The quality of the image capture is calculated using algorithms which,for example, analyze for sharpness and or contrast, or other parametersindicative of quality and suitability for further biometric processing.The system of the invention can use algorithms looking for an absolutemeasure of eye focus, since an eye has some generic features in commonacross large populations, or for a peak in the focus measure as imagesare acquired over the range of focuses scanned.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features, and advantages of embodiments arepresented in greater detail in the following description when read inrelation to the drawings, but not limited to these figures, in which:

FIG. 1 is a flow chart illustrating a system of the invention.

FIGS. 2-5 are graphical representations of the relationship between thefocus point of a sensor, distance between a moving subject and thesensor, and a fairly constant depth of field as images T1, T2, . . . Tnare acquired over time with examples of face/eye, iris, and status atdifferent image acquisition times Tx, each figure illustrating adifferent focus pattern.

FIG. 6 is a graphical representation of the improving quality of irisimages stored in the list over time.

DETAILED DESCRIPTION

While the invention is capable of many embodiments, only a fewillustrative embodiments are described below.

Referring first to FIG. 1 illustrating a process flowsheet according tothe invention, the process begins with a module 100 that determineswhether Acquisition for a particular subject should be started. Thismodule 100 may comprise several components depending on the specificapplication. For example the module may consist of a motion detectormodule, or a trigger that a previous subject has successfully performeda transaction with the system.

Upon initiating the acquisition, a local list of successively betterimages from the prior subject is cleared 101 in preparation for the nextsubject.

An image is then acquired 102 using a camera system. A camera system isused that can either capture images synchronously at a constant rate, orasynchronously on request by a computer-controlled trigger signal. Asdiscussed later, the camera may be operated at a variable acquisitionrate depending on the results of previous processing.

A Quality Metric module comprising, for example, one or more of thefollowing sub-modules: face detector, eye detector, focus measurement,iris area detector is used 103 to measure the quality of each acquiredimage in sequence when sufficient computing capacity is available butnot necessarily simultaneously with image acquisition. As discussedlater, one or all of these modules may be performed at a particular timeinstant depending on the results of previous processing. The qualityanalysis and selection system of Martin et al in US 2008/0075335, supra,which is hereby incorporated by reference in its entirety, is onesuitable Quality Metric system 103 for the purposes of the currentinvention, with the additional feature of the present invention whereinonly the best or a small, limited number of the highest quality of theacquired images is stored in memory.

An Acquisition Stopped module 104 is to perform an Acquisition Stoppedroutine. This module 104 ensures that the overall process is not beingperformed unnecessarily if, for example; the subject has walked awaywithout any data being acquired. The Acqusition Stopped module mayconsist of a time-out counter that compares to a threshold thedifference between the current time and the time that the Acquisitionprocess was started. The process for a particular subject can beterminated 109 or the last image can be stored 107 if a better 103 imagethan the best quality image stored at 110 is calculated.

A Comparator module 105 then compares the results of the Quality MetricModule with the results stored in a Local List in storage module 110. Inthe first iteration of the process, there will be no data in the LocalList in storage module 110. However, after several iterations, some datamay be present within the Local List 110. If the results of the QualityMetric Module 103 are greater than any of those on the Local List 110,then the imagery data is stored on the Local List, Storage may compriseappending the imagery data to the Local List 110, or may comprisereplacing 107 imagery data on the Local List that has a lower QualityMetric 103 value.

Step 108 is optional, as indicated by the box shown with broken lines.In certain embodiments where step 108 is absent, additional imagery isacquired automatically without changing focus values but is ratheracquired at a fixed focus, the quality of imagery depending on the exactlocation of a moving subject within the capture volume at the timesuccessive images are acquired. In certain other embodiments when module108 is present, the focus setting of the camera acquisition system isindependently modified prior to acquiring the next image. Severalmethods for modifying the focus setting can be employed as discussedlater.

After the focus has been modified, then imagery is once again acquired102 in the next iteration of the process.

The process continues until 109 either the timeout condition describedabove occurs, or the Quality Metric 103 exceeds a value.

Referring now to FIG. 2 the top illustration shows the disposition of anunconstrained subject over a period of time at times To through T6,showing that the subject may turn his head, or blink, for example. Thesolid, dark line in the bottom of FIG. 2 shows the disposition of thesubject's distance from the camera acquisition system. Note that thesubject is moving closer then further from the camera sensor in a randomfashion due to their relaxed disposition or inability to remain exactlystationary. The dotted line shows the disposition of the Focus Settingposition at different time instants. In this case, the Focus Setting hasbeen set to follow a sawtooth waveform over time. The small verticalbars on the dotted line indicate the depth of field of the sensor. Ifthe depth of the subject intersects any point within the small verticalbar, then the subject is in focus. The “Status” row at the top describesthe status of the subject with respect to the image acquisition system.For example, at T=T0, the subject's head is turned and no face isvisible. At T=T2, the subject's depth intersects with the depth of fieldof the particular focus setting at that time, however the subject'seyelid happens to be closed at that point in time. At T=T3 on the otherhand, the subject's eye is present, the eye is at least partially openso that the resultant Quality Metric has a finite value, albeit a lowerthan optimal value since the image is slightly out of focus. The imageryat T=T3 is therefore placed on the Local List. At T=T5, the subject'seye is present, the eye is at least partially open so that the resultantQuality Metric has a finite value, and the subject's depth intersectswith the depth of field of the particular focus setting at that time sothat the Quality Metric has a higher value compared to that of the imagethat is already on the Local List, and therefore the image at T=T5 iseither placed on the Local List or replaces the existing image on theLocal List depending on the particular embodiment of the invention.

FIG. 3 shows another embodiment of the invention with a different focussetting routine. The subject's disposition is as in the previousexample, but the camera acquisition module has the capability ofperforming rapid data acquisition over short time periods, upon certainconditions. Rapid data acquisition is not performed all the time sinceit is prevented by limitations in bandwidth and processing speed. In theembodiment shown in FIG. 3, the selected conditions for performingshort-duration rapid data collection for a fixed time period (in thiscase from T=T3 to T=T6 is the detection of a face, an eye, an iris thatis open, but blurred. If most of the criteria for successful acquisitionhave been met, then there are only very few additional criteria thatneed to change before valid iris data can be acquired. It is thereforemore probable than at other time instants that valid iris data may soonappear. The rate of data acquisition is therefore increased in order tobe ready to capture more iris data than would have otherwise beencaptured.

Referring now to FIG. 3, the thick vertical lines around T=T5 shows that4 images were acquired around this time period during the rapidacquisition mode, rather than just 1 image in the prior embodiment.

Referring to FIG. 4, the subject is moving generally towards the camera,in addition to random movement. In this case the focus setting is acombination of an auto-focus value computed from the average focus ofprior settings, as well as a sawtooth waveform as described in the firstembodiment. In this case, valid iris images are stored on the Local Listat T=T3, T=T5 and T=T6.

FIG. 6 is a graph showing on the Y-Axis the Quality Metric value ofimages as they are placed on the Local List over a short time period.Typically, imagery is typically placed on the list rapidly, but then asmore data is placed on the list it becomes more difficult and thereforetakes longer for new imagery to exceed the existing Quality Metrics onthe list. An example Quality Metric is Q=F (A+delta), where F is a focusmeasure where high values of F indicate more focused imagery and A isthe estimated area of the iris. Various known, alternative methods forsegmenting the iris and extracting the area and quantifying focus can beused.

The method is highly effective in many respects. A first advantage ofthe invention is if the disposition of the subject is immediatelyamenable to successful data acquisition (e.g. eyes are open and theirface is facing the system), then the system will acquire iris imageryvery rapidly. There are many methods for detecting the presence of aneye. For example, the Hough Transform disclosed in U.S. Pat. No.3,069,654 can be configured to locate circular segments of the eye dueto the iris/sclera boundary and the pupil/iris boundary.

However, if the subject is fidgeting or unable to remain stationary, oris distracted by baggage or children for example, then the acquisitionsystem will still acquire imagery, although it might take a slightlylonger period of time. However, the acquisition time for an amenablesubject will not be penalized by the system's delays in acquiring datain the case of a less amenable subject. This is crucial when subjectthroughput is considered. This is to be contrasted with systems that mayacquire and store a large number of images and then perform processingon the images to select imagery.

A second advantage of the invention is the ability to acquiresuccessively better iris imagery. In the current art, iris imageacquisition systems typically have resulted in the output of one imageof the iris deemed to have a quality suitable for matching, usuallyexceeding a threshold. If such an image is not found, then no iris datais captured. The problem with the current art is that there are someapplications when there will not be a second chance to acquire betterdata since the subject has gone elsewhere or is fed up with using thesystem. Ironically, however, the iris imagery they presented may havehad plenty of information for the particular application at hand. Forexample, if the image acquisition system is to be used to gain entryinto a house with only 100 subjects, then some of the iris imageryacquired earlier in the acquisition process may be sufficient.

A third advantage of the invention is the efficient use of memory, whichis significant especially when an embedded device is used. The LocalList contains only iris imagery that is successively of better qualitythan the prior imagery, and does not contain the imagery that wasoriginally acquired. In addition, depending on the application, theLocal List can comprise a single image which is replaced each timeimagery of a better quality is detected. After processing is complete,then the resultant image remaining in the Local List is the imageryacquired of the best quality.

In one embodiment, the invention obtains in-focus images by using afocus controller component that controls the lens to focus atsuccessively different points within a focus range, such scan controlperformed without any input from measurement of whether the image is infocus or out of focus, be it based from measurements of the image orother distance metrics to the subject. In terms of focus scan speed andhow it relates to frame rate, exposure time these relationships andrelated algorithms are known to those skilled in this alt.

Even when a subject is trying to stand still, there will be residualmotion. The system in some embodiments can increase or decrease the rateof image capture at different focuses in view of the degree of motion ofthe subject.

The system acquires a varying number of images, to account for the factthat in some cases we may acquire a good image on the first imageacquisition, but in other cases may have to wait for 10 or 20 imageacquisitions or more. If the system simply fixed the number of imageacquisitions to be 10 or 20, then we would dramatically slow down theaverage time it takes to use the device, and therefore reduce thethroughput of people using the device, since the number of imageacquisitions acquired would be set at the worst case, rather than beingadaptive based on the quality of the iris.

It is not good enough to have the focus set at the correct focaldistance opportunistically since, for example, the subject may blink orturn away even though the image is in focus.

If 10 or 20 or more images are being acquired, storing them can take upa lot of memory, which can be expensive in an embedded device. Thesystem of the invention successively checks whether the iris imagequality is better than the best iris image stored previously and only inthat case does the system store it. Alternatively the system canoverwrite the best iris image acquired so far to replace it with thebetter image. In this way, the system always has the best possible irisimage stored without having to use extensive memory. If the subjectturns away and the system loses its opportunity to ever again acquireiris data of a subject, the best possible image, even if not of highquality, will be stored and such image may have sufficient quality forbiometric identification under the circumstances.

In addition to the area to which the camera is pointed, we also cancontrol a focus control system such that a capture volume is sweptthrough. Unlike autofocus which requires settling time, and manydiscontinuous stop/start steps that eventually can wear down componentsand can take time to respond, we simply sweep through a focus volumerapidly, in order to opportunistically acquire biometric imagery.

While the invention has been described and illustrated in detail herein,various other embodiments, alternatives, and modifications should becomeapparent to those skilled in the art without departing from the spiritand scope of the invention. The claims should not be considered limitedto the illustrated embodiments, therefore.

We claim:
 1. A method for dynamic control of biometric imageacquisition, the method comprising: (a) acquiring, by a sensor, an imageof an iris of an unconstrained subject, without input from measurementof whether the image to be acquired is in focus; (b) determining, by adetection component, at least one of sharpness and contrast of theacquired image; and (c) determining, based on the acquired image,whether to trigger processing of an additional image of theunconstrained subject, and (d) if the additional image is to beprocessed, determining a time instance for triggering the processing ofthe additional image.
 2. The method of claim 1, comprising configuringthe sensor for operation with a single focus lens or using a singlefocus setting.
 3. The method of claim 1, wherein (b) comprises measuringat least one of sharpness and contrast of the iris on the acquiredimage.
 4. The method of claim 1, wherein (b) comprises determining atleast one of sharpness and contrast of the acquired image based on oneof: biometric matching using the acquired image, and detection of afeature of the subject's eye on the acquired image.
 5. The method ofclaim 1, wherein (c) comprises determining whether to acquire theadditional image of the unconstrained subject.
 6. The method of claim 1,further comprising determining whether to store at least one of theacquired image, and the additional image, if acquired.
 7. The method ofclaim 1, further comprising determining whether to store the acquiredimage or the additional image.
 8. The method of claim 1, furthercomprising determining whether to overwrite a stored image with theacquired image or the additional image.
 9. The method of claim 1,further comprising determining whether to initiate acquisition of aseries of additional images within a predefined duration of time. 10.The method of claim 1, wherein (d) comprises determining the timeinstance for triggering the processing of the additional image, based onan estimated degree of motion of the subject.
 11. A system for dynamiccontrol of biometric image acquisition, the system comprising: a sensor,acquiring an image of an iris of an unconstrained subject without inputfrom measurement of whether the image to be acquired is in focus; and adetection component, determining at least one of sharpness and contrastof the acquired image, determining, based on the acquired image, whetherto trigger processing of an additional image of the unconstrainedsubject, and if the additional image is to be processed, determining atime instance for triggering the processing of the additional image. 12.The system of claim 11, wherein the sensor is configured for operationwith a single focus lens or using a single focus setting.
 13. The systemof claim 11, wherein the detection component measures at least one ofsharpness and contrast of the iris on the acquired image.
 14. The systemof claim 11, wherein detection component determines at least one ofsharpness and contrast of the acquired image based on one of: biometricmatching using the acquired image, and detection of a feature of thesubject's eye on the acquired image.
 15. The system of claim 11, whereindetection component determines whether to acquire the additional imageof the unconstrained subject.
 16. The system of claim 11, whereindetection component determines whether to store at least one of theacquired image, and the additional image, if acquired.
 17. The system ofclaim 11, wherein detection component determines whether to store theacquired image or the additional image.
 18. The system of claim 11,wherein detection component determines whether to overwrite a storedimage with the acquired image or the additional image.
 19. The system ofclaim 11, wherein detection component determines whether to initiateacquisition of a series of additional images within a predefinedduration of time.
 20. The system of claim 11, wherein detectioncomponent determines the time instance for triggering the processing ofthe additional image, based on an estimated degree of motion of thesubject.